Report on the Evaluation of EVS Usage and Trends at the University of Hertfordshire : February to June 2014

The Electronic Voting Systems (EVS) evaluation project for iTEAM has investigated the current level of engagement in the use of EVS across the institution in 2014. It has built on the work and outputs of the JISC supported Evaluating Electronic Voting Systems (EEVS) project in 2011-12 and the work of the iTEAM project through 2011-2013. It offers an up-to-date examination of the trends in EVS adoption and the breadth and nature of EVS use across the different academic schools. The project adopted a mixed-methods approach to evaluate usage and engagement. The starting point was a desk study to examine the existing data on numbers of EVS handsets purchased by academic schools in 2011, 2012 and 2013 and registered across the University and to explore the details from the School reports previously submitted to iTEAM. Sources of data included Information Hertfordshire and the iTEAM archive. Quantitative surveys were drawn up and information requests for student numbers were made to Senior Administrative Managers (SAM). A series of interviews were held with School-based academics including EVS Champions and Associate Deans for Learning and Teaching. Three purchasing trends for EVS handsets by different Schools were found:- slow decrease in HUM, LAW and PAM, moderate increase in BS, EDU and HSK and rapid increase in CS, ET and LMS. In terms of levels of EVS usage in 2013 -14 four different patterns emerged among the schools. These showed: slow increase (CS, LMS and PAM), slow decrease (BS, ET, EDU and HUM), rapid decrease (LAW) and no change (CA and HSK). The EVS purchasing and usage trends comply with the figures given by Rogers for his technology adoption model. Some schools are characterised by successful ongoing EVS use over several years while other school strategies for EVS, which had showed promise early on, have faltered in their use. There was some evidence that academics in STEMM subjects are more likely to engage willingly with EVS use where larger groups are taught, but this is not yet in evidence across all the STEMM groups at this university. Furthermore good practice exists and flourishes across non-STEMM subjects as well. The strategies for successful School-based EVS embedding and continued use include the following three hallmarks:- •Top-down management support for purchasing of handsets and including training for academics and administrators, and alignment with the School teaching and learning strategy. •The existence of a core of innovators and early adopters of technology including the local EVS champions, who are willing to actively engage with their fellow colleagues in sharing the potential of EVS technology. •An engagement with the pedagogical implications for changing and developing practice that the greater use of formative or summative polling and questioning requires. The immediate future of classroom technologies such as EVS offers two main directions. Firstly, there is the continuation of adopting ‘institutionally provided’ handheld devices. This is a low-cost method that can be used easily and flexibly. The other options for classroom polling rely on sufficient wifi availability in the teaching rooms and/or mobile phone signal strength/network availability and capacity. It is anticipated that the capacity issue will present fewer barriers for adoption in future, and that the future of the classroom response systems is inevitably linked to the widespread use of mobile technologies by students

Planning for Success in Introducing and Embedding Technology to Enhance Learning

The authors reflect on the outcomes of recent change management projects for introducing technology into Higher Education in the UK and discuss key aspects which have led to success in the increasing use and subsequent embedding of learning technologies in the classroom. They focus on three areas where it is suggested that institutions need to ‘get it right’ in terms of justifying the expensive introduction of technology into the learning environment: the building and maintaining of the technical infrastructure; the provision of appropriate initial and continuing user support, which includes relating the use of technology to pedagogy; and the management of the impact of change on those who are faced with adapting to different ways of learning and teaching. These are mapped to a set of critical success factors by the authors. The paper investigate these firstly, via a case-study within a technology-focussed university, where its commitment to the enhancement of the student experience through using technology to support assessment and feedback mechanisms has increased. The authors explore how academics were encouraged to become further engaged within the process. Consequently, the use of technology in the classroom was no longer seen as being the preserve of a group of ‘enthusiasts’ or ‘early adopters’ but was perceived to be relevant to a wider user group. A further case-study shows how the critical success factors were applied to develop a flexible learning module within a more traditional teaching environment. This paper explores the importance of balancing underlying pedagogical approaches to the introduction of new technologies. It is proposed that while technology can be an excellent tool it should not drive the pedagogy. The aim finally is to ensure that throughout and following a period of change both academics and students can benefit from the appropriate use of technology to enhance learnin

Simple e-learning semantic tool

Copyright © 2010 Stanford Center for Biomedical Informatics ResearchIn this paper we introduce a simple e-learning tool (SET) that can be used by students, teachers and librarians to build, evaluate, maintain and test knowledge of an arbitrary domain terminology. The core of the tool is a conceptual model developed in Protégé. The knowledge base used for testing the tool is from the Business Strategies domain and it contains more than 10,000 definitions from domain experts. The main advantages of using Protégé were in providing a valid and consistent structure for the initial set of largely unstructured data and in generating interfaces for the tool implementation. Compared to the other on-line glossaries, the emphasis here is on quality, by using domain expert knowledge; on diversity of that knowledge (a typical term will be accompanied with two or more definitions from different sources) and on simple to use additional e-learning features such as definition search, essay “helper”, visual domain explorer, test generator, etc. The related work has mainly been focused on the following areas, namely - development of domain ontologies, development of ontologies of learning resources 1 (e.g. Chimera’s Project Delta, as described in Gardner, 2007) and development of on-line glossaries (e.g. Merriam Webster Online ) and encyclopaedias (e.g. Wikipedia) . Our work differs from each of the above mentioned in that it builds on development of a general and reusable glossary ontology that uses as instances, definitions taken from domain textbooks and refereed journals. In addition to that, we provide an application built on top of this ontology that can be used as a cognitive support tool

Mapping the technology landscape : linking pedagogy to the affordances of different technologies

This work evaluates the application of different learning technologies and their suitability to support blended learning approaches in Higher Education. Chickering and Gamsons's Seven Principles for Good Practice in Undergraduate Education (Chickering & Gamson, 1987) were used as an underlying pedagogical framework to evaluate the "perceived affordances‟ (Norman, 1999) of learning technologies.Chickering and Gamson‟s principles were selected as a framework due to their "face-validity‟, the accessibility of their language and since they have been derived from numerous years of reflective and effective teaching.Along with the principles we describe and recommend an innovative methodology for evaluation. This methodology can be used in a context of similar evaluation exercises.Final Accepted Versio

The Long March or Bold Strokes : Comparing Strategies for Adopting EVS Learning Technology at a UK University

In recent years there has been much encouragement to investigate the use of classroom technologies to enhance the student learning experience especially in the STEM subjects but now extending across other subject areas as well. A typical classroom technology is electronic voting system (EVS) handsets which allow a lecturer to invite students to vote for their choice from a selection of given answers. Recently, a medium-size UK University has purchased over 9000 EVS handsets for use across the academic Schools as an innovative means for supporting formative and summative testing. Numerous training and support sessions have been provided to staff with the intention of supporting new and experienced users and increasing the take up by academics. As noted in earlier research reported at ECEL 2013, the student feedback was very positive for the use of EVS for formative activities, and less so, for its use in summative assessment. A recent review of the trends of EVS adoption at the University has been undertaken to inform decision-making and future use and support for the technology. One aspect of this review has considered the effectiveness of the strategies adopted by different academic Schools. EVS adoption and use across the University has been compared and placed within Rogers’ theory of the diffusion of innovation. This paper further considers a set of six different strategies adopted for EVS use by academic Schools. They have been categorised according to several variables, including their choice of speed of uptake and the number of handsets in use. The inherent strengths and possible weaknesses of the approaches adopted are considered. Among the questions raised were, does a strategy of large-scale technology adoption over a short time period indicate a greater likelihood of long term engagement and ultimate adoption of the technology? Or, does a longer elapsed time taken for a gradual purchase and adoption of EVS technology suggest a greater inclination for the embedding of technology for enhancing learning? What other success factors should be considered alongside the training and support provided for technology adoption to enhance the likelihood of long term adoption of classroom technologies? The discussion provides a comparison of six different strategies identified across the university and the rationale behind them and then proposes a set of strategy choices which can lead to a greater likelihood of successful adoption of classroom technology

The benefits and challenges of large-scale deployment of electronic voting systems : University student views from across different subject groups

This document is the Accepted Manuscript version of the following article: Marija Cubric and Amanda Jefferies, ‘The benefits and challenges of large-scale deployment of electronic voting systems: University student views from across different subjects groups’, Computers & Education, Vol 87: 98-111, September 2015, doi: http://dx.doi.org/10.1016/j.compedu.2015.04.004 This manuscript version is made available under the terms of Creative Commons Attribution-NonCommercial-NoDerivatives License CC BY NC-ND 4.0 (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.Electronic Voting System (EVS) is a classroom technology that provides a means to increase students’ engagement, attention and attendance. The purpose of this paper is to provide a deeper insight into students’ views on the benefits and challenges of EVS in the context of a large-scale institutional deployment and across different subject areas in higher education. The data were collected from an online survey of 590 students across eleven academic schools at a UK university. The non-linear principal component analysis of 32 question items from the survey showed that learning benefits, classroom-related benefits, usability and student-centered challenges are four distinctive dimensions in students’ perceptions of the use of EVS. The non-parametric group comparison tests suggested that there are significant differences in learning benefits and challenges across different subject groups. However, the disparity appears to be related more to the way the EVS was used and the experience of students with it, rather than resulting from disciplinary differences. Content analysis of open questions revealed that summative use and staff competencies are the main issues related to EVS use by students. Finally, despite the overwhelming perception of the ease of use, it was found that usability could be an issue for students when EVS was used for summative assessment. The implications of the study are: for practitioners, it underlines the importance of the focus on formative benefits of EVS as only then and regardless of disciplinary differences, can the promised rewards of the technology be gained; for institutions, it outlines some of the new challenges specific to the large-scale institutional implementation, judged through the lens of students’ experience; for researchers, it provides an overview of the literature on large-scale deployment of EVS and it suggests some new areas for research on the use of EVS in higher education.Peer reviewedFinal Accepted Versio

Design and evaluation of an ontology-based tool for generating multiple-choice questions

© 2020 Emerald Publishing Limited. This accepted manuscript is deposited under the Creative Commons Attribution Non-commercial International Licence 4.0 (CC BY-NC 4.0). Any reuse is allowed in accordance with the terms outlined by the licence, here: https://creativecommons.org/licenses/by-nc/4.0/. To reuse the AAM for commercial purposes, permission should be sought by contacting [email protected]: The recent rise in online knowledge repositories and use of formalism for structuring knowledge, such as ontologies, has provided necessary conditions for the emergence of tools for generating knowledge assessment. These tools can be used in a context of interactive computer-assisted assessment (CAA) to provide a cost-effective solution for prompt feedback and increased learner’s engagement. The purpose of this paper is to describe and evaluate a tool developed by the authors, which generates test questions from an arbitrary domain ontology, based on sound pedagogical principles encapsulated in Bloom’s taxonomy. Design/methodology/approach: This paper uses design science as a framework for presenting the research. A total of 5,230 questions were generated from 90 different ontologies and 81 randomly selected questions were evaluated by 8 CAA experts. Data were analysed using descriptive statistics and Kruskal–Wallis test for non-parametric analysis of variance.FindingsIn total, 69 per cent of generated questions were found to be useable for tests and 33 per cent to be of medium to high difficulty. Significant differences in quality of generated questions were found across different ontologies, strategies for generating distractors and Bloom’s question levels: the questions testing application of knowledge and the questions using semantic strategies were perceived to be of the highest quality. Originality/value: The paper extends the current work in the area of automated test generation in three important directions: it introduces an open-source, web-based tool available to other researchers for experimentation purposes; it recommends practical guidelines for development of similar tools; and it proposes a set of criteria and standard format for future evaluation of similar systems.Peer reviewedFinal Accepted Versio

EXECUTABLE ARCHIVES: Software integrity for data readability and validation of archived studies

© 2021 author(s). The text of this paper is published under a CC-BY license (https://creativecommons.org/licenses/by/4.0/)This paper presents practices and processes for managing software integrity to support data archiving for long term use in response to the regulatory requirements. Through a case study of a scientific software decommissioning, we revisit the issues of archived data readability. Established software lifecycle management processes are extended with archiving and data integrity requirements for retention of data and revalidation of data analyses. That includes the software transition from operational to archival use within the Executable Archive model that extends the traditional data archive with computing environments with software installations required to reproduce study results from the archived records. The content use requirements are an integral part of both data access and the software management considerations, assuring that data integrity is fully supported by the software integrityPeer reviewe

The Impact of Cytoplasmic Inheritance on Sperm Quality in Fleckvieh Bulls

Detrimental impact of certain mitogenome mutations on sperm quality traits, and consequently on male fertility is well documented in humans. With a quantitative genetic mixed model, we analysed the impact of cytoplasmic effects, maternal lineages treated as random effect, on sperm quality traits in 554 Austrian Fleckvieh bulls. We have observed that 2% of the phenotypic variance for transformed total number of spermatozoa is due to cytoplasmic (maternal lineage) effects. Regarding percent of viable live spermatozoa, no cytoplasmic effects were detected. However, the observed effects still need to be further evaluated from three perspectives, the analysis of the mitogenome polymorphism effects and the impact of the mitogenome effects on the realised fertility as well as on the whole production economically

The Impact of Cytoplasmic Inheritance on Sperm Quality in Fleckvieh Bulls

Detrimental impact of certain mitogenome mutations on sperm quality traits, and consequently on male fertility is well documented in humans. With a quantitative genetic mixed model, we analysed the impact of cytoplasmic effects, maternal lineages treated as random effect, on sperm quality traits in 554 Austrian Fleckvieh bulls. We have observed that 2% of the phenotypic variance for transformed total number of spermatozoa is due to cytoplasmic (maternal lineage) effects. Regarding percent of viable live spermatozoa, no cytoplasmic effects were detected. However, the observed effects still need to be further evaluated from three perspectives, the analysis of the mitogenome polymorphism effects and the impact of the mitogenome effects on the realised fertility as well as on the whole production economically